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The Role of Hoesviremcat and Evaluation “EPARTMrjNT of 

in Instructional Technology this docSK HAiTel^'wpflODucED 

EXACTLY AS RECEIVED FROM THE PERSON OR 
T? C|,...annr-.v* ORGANIZATION ORIGINATING IT POINTS OF 

iCXCnarcl t*. opencer view or opinions stated do not neces- 

sarily REPRESENT OFFICIAL OFFICE OF EDU- 
CATION POSITION OR POLICY 



There are, in e.>:istcncc, innumerable reviev/s of the research dealing with 

instructional technology (bumsdaiue, Torkelson, Allen and l'’inn) * Included under 

the rubric of 5,nstructionall technology are educational television, computer 

assisted instruction, programuied materials, teaching machines, simulation games, 

instructional films, f jJjnstrips, slides and other mrxterials, textbooks, etc. All 

0 * . '■ 

such variablesS, as well as many more, since the introduction of the hornbook in 
the 16th century, have been researched, compared, evaluated, accepted, rejected, 
or ignored. Much of the research has been temporal or cyclical in nature; empha- 
sis occurs on one topic, or another as a specific piece of equi.pment is made 
available in the market place, or as federal funds rev7c?.rd certain types of 
innovation. 

It is not the purpose of this paper to review specific re.scarch in the area. 
It is assumed that the reader car find such material elsewhere. Here, the pur- 
pose is to attempt to make an editorial comment on the field as a whole, to indi— 
cate some sources of assumptions, causes of the f5.ndings, the role of measurciaent 
and eva3.uation in the area, and a look to the future of educational techno3.ogy. 

If one looks at the research, one can easily estimate the populaixty of 
various topics of concern. Discussions of the Big Three-^Television, Compucei 
Assisted Instruction (CAI) , and Programmed Instruction (PI) compose a majority 
of the V 7 ritings in ^’he area. In general, one finds that the early research 
effox'ts in each area tend to be focused on the hardware; x.e. Can TV Teach? Do 
teaching machines v 7 ork? Later, concentration turns to tbeoietical developments, 
and more system or operational considera-txons. To a considerable extent., hov*7 
ever, the hardware systeims of instructional technology have been used J.n some 
operational settings with little regard for research findings. 



ERJ,C ■■' Richard E. Spencer is head of the Tff ^sur<-m«nt and Research Division, University 
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A3J, of the otuflles have acme basic assumptions, but it should be apparent 
that there is a basic prob!!,em v/ith instructional technology research and use 
vjhich has been generally ignored. In the first place, the definition of instruc- 
tional technology tends to emphasize hardware, generally to the exclusion of 
Instructional content, or teaching strategies or methodologies. These iteras are 
variables which exist on ETV, or films, or are included in PI, textbook writing, 
or any other presentation of instruction. Publications seem to hcive been concen- 
trated on the assumption that the hardx^are can do something unique, or operate 

* 

indepel^dently of what someone puts into it. Such research normally results in an 

all too common finding in this area — no significant difference. The hardware 

(the media end of Instructional technology) as used, results in learning to about 

the same level as a real, live instructor does, 

« 

As one progresses from concentration on the hardv7are to what is performed 
with the hardware (sequence of learning, audio-visual stimuli, reinforcement, 
overlearning, etc.) one finds small but statistically significant differences 
between experimental and control groups. Unfortunately such gains are usually 
temporary and disappear upon delayed ;recaXi evaluation. Thus, the general state 
of the art can be defined in either an optimistic or pessimistic way: 



Instructional technology can teach just as well as regular, conventional 

% 

methods ; or 

instructional technology can teach no better than regular, conventional 
methods . 



Thus far, however, we have not defined "v/hat" is taught, how something is. 
taught, or v?hat learning is considered to be. Secondly, vjc have not concerned 
ourselves with student "attitudes" toward the hardware style of teaching. Most 
of the effort in instructional technology has been put into one-shot programs, 
often developed for research use only, vjhich rarely see the light of day iii a 
continuous on-going instructional. opera.tion. Those that do devexop Ixilo 
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operational instructional programs sliow no better results (in achievcnaent) than 
does regular ins true tioii, and the students generally dislike it more. Pv.eports of 
non-invoivemeiat > separation from the instructor, uninteresting materials, etc, 
are common for hXV; for CAI and Programmed Instruction the most common student 
response is that it is boring, What is the basis for these rc\ther discouraging 
results? Is instructional technology doomed to yXay only a supportive role in 
American education? Why hasn't instructional technology fulfilled its potential? 

I 

Some of the assumptions underlying the use of instructional technology could 

stand critical inspection. There is a considerable "religiosity" associated v;ith 

Instructional technology— those that are in the field seem to believe that the • 

potential is just lying there waiting to be tapped. With only a slight degree of 
0 * 

effort, the technology will prove its worth. Faith is built into many research 
designs, but satisfaction occurs with no significant regularity. An evident 
assumption appears to be tliat the media is in fact the message (McLuhan and 
Fiore). Xt is certainly true that one can study content, as a student, only 
according to vzhat content is available. What university courses exist define for 
the student what he may study — he is, in part at least, led through an a priori 
system of content, "...the students rate of learning is restricted to the 
teacher’s rate of presenting; the amount to be learned by the student is limited 



by the amount the teacher presents; and the pattern of instruction by which the 
student is supposed to learn is determined by the pattern the teacher decides to 
use for the whole class," (Dial-Access, p. 5) Similarly he may only read from 
the reservoir of v;hat has been v/ritten; he may see films only related to that 
which film makers consider worth filming. Practical applications of ETV, on col- 
lege campuses at least, have resulted in the video taping of hours and hours of 
the head and shoulders of some professor performing precisely the way he does in 
class." If the media is the message, the message on this TV media is that educa- 
tion is a bore. This attitude has become so fixed in the minds of the students. 



perhaps because oj: the rather obvious cornparisoa to conimercial TV, that order 
to obtain significant increments in learning v/ith ETV, one has to exert consi- 
derable effort, and consider many different variables and factors in the learning 
situation. 

A second apparent assumption is that one must concentrate on the INPUT to 

the instructional system rather than OUTPUT; i.e. the assumption is made that vze 

know v?hat the objectives of a course really are. The method by vzhich an instruc- 

# 

tional segment becomes involved V7ith instructional tecVmology is to start V 7 ith 

how it is done now— *-in the regular (i.e. conventional) classroom 5,nstructional 

setting. This methodology of approach may set the stage for the all too popular 

"no significant differences" that so often result. Instructional technology has 

been constrained by the already existing structural pattern, Knov7ledge, or con- 

♦ ' 
tent, has long since been carved up into segments, placed at levels, and defined 

by a rendition of- the Sapir-Whorf hypothesis. As Sapir states (Sapir, p. 578): 

"language, constitutes a sort of logic, a general frame of 
reference, and so molds the- thought of its Habitual users," 

« 

Such a theory may be applied to an educational system — one V7hich has defined 
reality and the real world (see Jerome), divided it up into language arts, arith- 
metic, social studies, physical education, etc.; housed it in a certain manner, 
jupplied the media (i.e. teacher and text), financed it, dated it (from September 
to June), timed it (8:30 to 3:15) to such an extent that the system becomes a 
reality as Benjamin so aptly points out in the Saber-toothed Curriculum. It is 
difficult to develop a independent developmental procedure, and a capability to 
stand aside and view the system, define its objectives and evaluate its outcomes 
without the constraint of the e^xisting structure within V7hich Instructional 
technology has been operating. The organization and the Institutionalization of; 
education itself has formed the greatest barrier of the use of instructional 
technology. The research on instructional technology has consistently occurred 
within the system — affected and controlled by the systemization already in 



existence. This system controls vjhat V7C may experiment upon, hov7 we may experi- 

menu, and how we are to treat the results. The potential for significant dif~ 

ferences may not existj or those things upon which V7e determine significant dif“ 

fercnces aren’t the important ones. The system itself may be the most important 

variable related to the future of instructional technology. 

Those variables on v 7 hich instructional technology research is conducted are 

largely based on those factors held dear by individual professors or teachers. 

An analysis of these variables can be conducted in tv70 V7ays"‘-(1)' ask V7hat educa~ 

tional objectives are, or (2) inspect the examinations used. The former approach 
0 • 

yields descriptions of everything from "ivnproving thinking” to ”reducii-\g the 
n\yQptic view of our culture.” The latter xndicates that most 0 xaminat.ions coii"* 
taiii a majority of factual, memory-type test items (essay or objective) tox^ard 
which learning must he oriented. 

The type of examination system used in an educational program represents to 
a considerable degree the ultimate objectives in that learning system. The exam- 
ination, therefore, serves to direct, influence, and determine V7hat is measured 

« 

and evaluated— -and thus the concentration by design, intent or accident, that the 
student v/ill place on certain elements to the exclusion of others— regardless of 
stated educational objectives. The examinations state the objectives very con- 
cretely, and with more meaniingfulness to the student, than does the instructor, 
course outline, or textbook. If the system is composed largely of factual 



material, that which will be learned will be factual, at the expense of atti- 
tudes, appreciations, or problem solving concepts and understandings. To the 
extent that the evaluative instx'uinents represent a knov7n goal, that which occurs 
• in the classroom (regardless of media or technology) is that which the student 
will learn (Spencer). The student himself controls what he V7ill pay attention to 
'or learnj partly on the basis of his own j.ntecest and motivat5.on, partJ.y (but 
only slightly) on the degree to which the instructional program can develop 






interest, partly on the banis of his u priori attitudes toward the content of the 

* 

instruction (he likes math hut hates English)} ^nd on the basis of what he sees 
as the V 7 ay ho can maneuver through the system. Whether technology is used or not 
is Irrelevant at this point-~-the overriding influence is the system. Any altera*- 
txon in the system, such as moving frora conventional instruction to some sort of 
instructional technology involves a relationship with an already existing insti- 
tutionallKation and rex^ard system. If these later variables are related to 
Instruction in such a manner that they rcv7ard convention more than nevrer proce- 
dures, the results of the nex7er procedures V7ill tend to, be. depressed irrespective 
of content. This effect can operate on content, faculty participation, iutejiest, 

and student response. An assumption underlying instructional technology experi- 
0 • 

mentation exists \7hich is relative to this area — that the system is adaptable to 

« 

instructional technology, and that operations jin' this area vrill be v7elc.ome» Such 
an assumption has not been wholeheartedly validated. 

As we have seen, examinations themse3.ves may be classified as axi instruc- 
tional technology, CAI and Prograiumed Instruction have emphasized this variable 
in their operations, by attempting to supply immediate, feedback of results to 
students as they x7ork through an instructional sequence, in providing positive 
reinforcement (getting the test Item correct), and in testing only that which has 
been presented. One major problem comparing instructional technology with con- 
ventional systems is the introduction of different examination variables into the 
picture. If one proceeds to- "program'' a course, and supply positive reinforce- 
ment and immediate feedback, one is varying a considerable number of factors 
other than the method of presentation (i.e, by computer, self-instructional pro- 
grammed text, or T.V. film or x-7hat:cver) . One is also introducing a methodologi- 
cal difference (in the psychological sense) j the manner in which the material is 
presented is different. One can find in the literature that changes in these 
methodological factors, x^ithin a conventional classroom system, can produce a 
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significant difference. Tl\e media may have little to do with learniiig — what’s on 
the media is the thing. 

There has been considerable empliasis on statistical treatment of ins true*" 
tional technology research, primari3.y because many of the assumptions under 3.ying 
the statistics used arc; not met by the research design (Lumsdaiue, p. 593). A 
second problem is a deterirdnation of significance for V7hat? What type of signi- 
ficance is one real3.y after? 

If V 7 C imagine ourselves setting up an educational enternriae from scratch, 
V7ith no previous institution ever having been in existence, V7e see that we must 
get the naive (student) connected to the all-knowing (professor). When we accom- 
plish this purpose we can note by tests of significance that the student/profes- 
sor system results in more learning (especially if the professor makes the exami- 
nations) than docs the lack of such a combination. Thus we conclude that IF V7hat 
the student/prof cssor system has developed is of v alu e to the society, voi recom- 
mend its continuance, elaboration, and extension. Some systematic education is 
better than random no education. Now if we want to experiment with different 
types of educational systemi?'.ation, we tend to prejudice our thinking in the 
direction of very slight changes (relatively), each wUhin the student/professor 
system. Let's put the professor on T.V. , or film, or radio; add a textbook, or 
program the instruction, or use a computer; se3.f-pace or time-pace the student, 
put him into homogeneous groups of slow, average, or fast learners, etc. The 
amount of potential effect these variables can possibly have upon student 

I 

learning is indeed small. The results validate this argument— but perhaps, our 
criteria have changed. The IF statements (if we can slip into computer language 
for a moment) have been changed. Now, wo may V7ant significance tests like IF we 
can do it cheaper, faster, to more people at once, V7ith less effort, etc., the 
significance of the results is tested by a valid "no significant difference," 
i.e. we are successful if there are no differences. If however we are interested 
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in the students learning more, we have other things to consider. Now v/e have 
stated a specific direction; v^e raust define ’’more” of v;hat, and our effort (time, 
money, personnel) is directly related to the extent of the ''more'' that we v?ant. 

If we decide to do in half the time (one semester) V 7 hat was qlone before (in two 
semesters), again no significant difference in criterion test scores is a signi~ 
f leant difference — our real criterion is time, which (2 X 1) is significantly 
different. But this design assumes we merely vrant to do the same kind of job. 
This is another primary problem V7ith our assuraptions . We assume that wc know 



what V7C need to get accomplished, and that the educators-’ know hov7 to (pedologi** 
cally) get it done. 

As McClenry states (in another context, but applicable here); 

1. "The Immediate applicability of research findings becomes a 
primary criterion in the identification of the problem, 
research design, and support." 

2. ‘'The field of investigation is limited prematurely and may 
exclude esseni:ial criteria and/or experimental variables." 

3. "The factor of self-correction-— -control and verification of 
activities and conclusions — is suspect." 

4. "The haste to state conclusions crystalizes thinking and 
precludes examination of assumptions," 

5. "Projects are too ambitious for the personnel and resources, 
thus they contribute to loose speculation about observations 

made in. different situations," (McCleary, p. 7) ‘ - 

Educators are constantly arguing about what is a good teacher, and what is a 
"good" method of instruction. We are concerned about how to evaluate when a stu- 
dent has reached success or mastery. Our examination, evaluation, or grading 
system again gets in our way. Is an "A" giv-en by a professor sufficient indica- 
tion of mastery? Is one "A" equal to another "A" in another course or subject? 

In instructional technology research, one notices that criterion tests tend to be 
rather easy (as most classroom tests are) . Most of the variance among students 
is determined by a spread of scores among the top 25% of the scale (between 75- 
100% items right). If we want to make comparisons, V7e obviously don't have much 



room to operate, A football team (on offense) doesn’t often get a long run for a 

I 

touchdown because most of the time they are very near the goal line and do not 
have very far to go. 

An ideal test for comparing differences betvzecn treatment groups V 70 uld be 
one which distributes the students around an optimum mean (midv/ay between chance 
score and the ma>:imum possible score). Thus, on an objective test with five 
alternatives per item, and 100 items, an optimum mean V70uld be the maximum score 



(100) minus chaiicc score (20) « 80. A range of 80 betv’een 20 and 100 v70uld put 
the optimum mean at 60. Most c3,assroom tests have means higher than this; and ■ 
thus, the room to shov 7 the differences is depressed artifically by the type of 
examination used. There is not enough range for the students to shov; the gains 
they may have made. 



In programmed instruction, and CAI, the tendency is to devise tests which 



have a mean score nc lower than 95% in order to retain the assumption of positive 
reinforcement. This theory is seriously open to question, both on theoretical 
and statistical grounds (if one x-7ants to compare groups) . Tests are approxima- 
tions of- criterion behaviors sought by an instructional program. One instructs 
ox* presents a specific type of content, to V7hich students respond. In an open 
situation (i.e. instructive) as traditional classroom instructxon is. learnings 
are permitted to vary two-dimensionally — in amount of material that is learned 
(horizontally on a sequence ladder) ; and in k ind (vertically across sets of lad- 
ders) Learning includes a variety of types, including factual knowledge or 
vocabulary (a neat, sequential, additive type of l''arning) , to appreciation (very 
Imprecise), In a closed, structured situation (P.I., C.A.I.) that V7hich is 

taught is more prescribed and defined. There is less opportunity for hori^-t-''‘-al 

* * 

movement. The two situations present very eJifferent instructional settings 



within v;hich V7ide variation may occur. 
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There is certainly a value to be gained by the use of instructional technology. 

It presents an. opportunity to investigate and possibly change an instructional 
unit or program and it exposes the system to viev7. This in itself may yield 
changes which benefit the student, but the one who is changing is most probably 
the individual faculty member; thus, not only does he change his procedures for 
the TV or C.A.X. or P.I. course, he also goes back and changes the conventional 
course, This system has not been investigated, because of a hypothesis that 
arranging a course for TV, for example, is sufficient reason to yield significant 
differences, 

The potential for instructional technology is far greater than its use to • 

t 

date. Although wildly expensive, for educational minds and pocketbooks, instruc- 
tional tochno3-ogy could be used to produce extens3.ve "significant differences." 

In the first place let us acknov/ledge that the forgetting curve for school 
learning is monumental — mos.t factual learning is forgotten with5.n a very short 
period of time and most school learning is factual. Instructional technology can 
be used quite effectively for providing a tachistoscopic-memory device that could 
almost guarantee set levels of memorization— even overlearning. One could flood 
the student^s perception with mechanical systems which he could scarcely escape 
from (1984 is only 16 years away). Attitudes, understanding, problem solving, 
evaluation, concept formations tasks are more difficult— but again readily pos- 
sible given the appropriate resources to, accomplish the task. If one looks at 
instructional technology installations across, the country, one is remf.nded of 
v/here data processing was first housed- — in basements, garages, and converted 
steam rooms. Present instructional technology setups indicate that instruc- 
tional technology has evi.deiitly not been accepted by society. As Coombs states. 

"Education’s technology, by and large, has made surprisingly 
little progress beyond the handicraft stage, V7hereas remarkable 
strides have been made in the technology and productivity of 
many other sectors of human activity, such as medicine, trans- 
portation, mining, communications, and manufacturing. " (p. 7 ) 
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Jackson agrees : 

"...there ±s reason to suspect that many of the bolder fore- 
casts concerning technological change in education will not be 
fulfilled," (p. 3) . . 

Coombs goes on to suggest four reasons which might contribute to this 
problem: . i, • 

1. "First is the sha r p increase in pop u lar aspirations fo r 
education . V7hich has laid seige to existing schools and 
universities." 

^2. "Second is the acute scarcity o f resources , which has con- . . 

• ' strained educational systems from respondix-ig more fully to 
new demaixds . "■ 

3 . "Third is the inherent inert ia of educational systems . 

^ which has caused them to respond too sluggishly in adopting 

their internal affairs to new external necessities, even 
\>;hen, resources have not been the main obstacle to adaptation," 

4. "Fourth is the inertia of societies themselves — the heavy 
vjcight of traditional attitudes...." 

Instructional technology offers the potential of communicating the necessary 
facts and information into the public domain, rather than keeping educational 
materials locked inside classroom v/alls. Confining ETV to closed-circuit is an 
exaExple of the degree to x-7hlch financial restrictions and poor planning eliminate 
wide sources of education from public view. Perhaps the public is not inter- 
ested; they certainly don't rush to watch NET, but the reason may be that the 
programs are 

_..l. -highly "arty". — drama, dance, music (and cooking) - - 

2 . highly academic, and by this I mean dismally boring and 
uninteresting for the potential audience. 

That the Beverly Hillbillies is more popular than NET should not cast blame on 
the populace, but on the programming of NET! 

Given the necessary and adequate fundihg, x>7ith creative and imaginative pro- 
ducers and v;riters , ETV can be as interesting and communicative as commerical TV. 
ETV has gone underground — perhaps because the public V70uld be horribly shocked at 



the state of teaching in American education, miy is there the attitude that V 7 C 
must not ’’entertain" our student audiences? Why are textbooks dull? There is 
simply no support, either financial or in the faculty reward system, for making 
an instructional program exciting and stiiaulating. There is no .system to revrard 
the innovator, and thus no particular reason to evaluate the results. Why eva.l~ 
uate if nothing different is going to occur? The academic institutions may be 
competent, but they may also be impotent, 

There are certain other variables which demand consideratiem 5.n regard to 
¥ 

instructional technology, particularly iia the area of s'pecific evaluation proce- 



dures and research designs used. Objective testing has been considered the e: 
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plificatiou of e.mpiricical research. Atkin (1963) for example indicated that 
innovation or creativity may develop beyond the capability of objective criteria 
to evaluate them. How then are the innovators V7ith instructional technology to 
calculate their effects? The question resides, unfortunately, in the case of 
objective criteria, or subjective judgment, with WHO does the evaluation, and HOW 
the criteria are arrived at. Obviously, if one pj.cks objective instx'uiuents which 
depress the capability for the measurement of creativity (x^hatever that is) , it 
will not be discovered in the treatment or experimental design. Similarly, with 
subjective judgment as the method, one is dependent upon the capability of indi- 
viduals to observe creativity. The criteria, however, remains constant in both 
cases— to develop a measuring instrument (be it observational, experlencial, 
objective, or performance) which will consistently (reliably) be sensitive to the 
behavior we are interested in developing. Secondly, it must also do the job the 
next time, or vzith different subjects, or different operators of the project. 

The development of an instrument alv; avs is a less than perfect approximation 
of the behavior \^e might desire to measure — ^.tests and people are sensitive and 
in sensitive. Care must certainly be applied in order to prevent the instrument 
from concentrating, or over-structuring the intent of a project, or determining 
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In advance the results one raay potentially render. If only a single » objective 
instrument is used as the criterion measure, we are placing all our marbles j.n 
one bag, A multl-media evaluation model (using objective tests, observational 
records, performance criterion, etc.) vzquld be more likely to discover changes in 
students than a single-media approach to evaluation. ^ 

The type of evaluation system employed also depends on what it is v^e are 
attempting to do. , If the object of the design is to shox7 that one system of 

t 

instruction is better thaii another, evaluation design, and media may differ coii" 

siderably from a project, which is determining xdiat is the best way to accomplish 

a specific objective, or an exploratory instructional system vyhich needs feedback 
1 

into the system as the instruction develops. 

In the same vein, much of the research vyith instructional technology has 
been concerned with a "defensive" research design; i.e. ETV is as good, or is 
-.better than conventional instruction (considering learning as the only criterion) 
A more rational approach to instructional technology vyould seem to be in the dir- 
ection of "How can instructional technology be used to improve instruction?. 

. , » 

VJheh, at what age, in i^yhat areas, and with what types of learning? (See 
Carpenter, 1968) 

It should be apparent from hov? instructional technology is used in education 
today that whole programs tend to be presented rather than single-variable sti- 
muli; i.e. French I in 6th grade is presented as an entity, rather than, say, the 
concept of "carrying in tx-yo-digit addition." There has been little effort to 
discover where a particular media or technology vyould be of help. In general, 
this is primarily due to the fact that: (1) v;e don't knox-y where certain proce- 
dures vyould help, and (2) vye don’t know x*7here, in our conventional educational 
system, the most help is needed. 

We do not normally include in our educational system a constant evaluation 
program to inform administration diagnostically xdiere the good and poor points of 
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the pvograiTi are Xocated. For example, can we ansirer questions like: (1) Is high 

school chemistry t^iught better than history? (2) Is long division learned to the 
same eKte.nt as multiplication? (3) Is the English they learn in high school 
enough? 

The university 3.evel is particularly vulnerable to questions of this sort 

# 

since there are few criteria for judgment. The graduate schools can make judg- 
ments about undergraduate education— but few undergradiuates go to graduate school 

I 

Since, however, this relationship is one of the few v?hich can lead to evaluations 
of undergraduate education, the lovzer level system tends, perhaps, to orient the 
progx'aiu in the direction of graduate school — even when only a minority of stu- 
dents ever arrive there. 

# ^ 

The college can evaluate high schools — but again not all high school stu- 
dents go to college. The high school can evaluate junior high and they, in turn, 
the elementary. In general this results in a system of constantly castixxg blame, 
downvzard, Each in turn sets its own criteria for selection and promotion on the 
ever present evaluation lying one step higher. The eva3.uation system is one 
which merely exists — no one thought it out, planned it, or adopted it — it grev 7 . 

Is it the proper one? If one x^ere to determine the practical objectives of the 
use of instructional technology, one x-jould have to conclude that the criterion 
are already built into the system — get more people into college or gj'aduate 
school! This has lead, one could presume, to an over-intellec.tualiisation of 
Instructional technology. From observation (if I may use Atkin ;s method of eval- 
uation) one observes the primary resources going into college preparatory 
courses, material pre-determined by already existing course and curricular struc- 
ture, required rather than elective courses, etc. Very little has been done x^ith 
motivation, self-evaluation, vocational education, visual communication, inter- 
personal cooperation, selection and interest, reading cross-field educational 
designs, etc. We sCill operate in the structure developed a hundred, years ago — 
the assumption must be that this is the way it ought to be done. 
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What i.ustruci;ional techuoXogy is used, 



hov7 it is used, 



and on what is it 



applied “...is gu5.ded largely by force of personal conviction, and evaluation 
research is mainly of use in helping consumers decide v/here personal convictions 
to buy.” (Bareiter, 1967, p. 192) 

VJith the advent of instructional technology, some institutions have been 



altered in \io.ys they had not foreseen, Fi3.ra usage requires projectors, screens, 

« 

and projectionists, as v^ell as some delivery method for getting the supplies to 

where the students are. C.A.I. requires cables, a student station or more, a 

computer.* 'TV requires a channel, a. tower, a studio, etc., etc, To the extent 

, * *< 

that such mechanics- are bought, or subscribed to, the students receive a differ*^ 



ent sort of instruction. The instructor ma 3 >‘ be a machine and not a person; it 

may be a self-controlled rather than teacher structured, vrrltten not spoken, or 

presented by a stranger on a screen rather than a teachei; within the womb of the 

\ 

classroom. It is obvious that the individual teacher, bkerefore, looses direct 
control of what is presented, how it is presented, and when it is presented. The 
teacher becomes more of a participant than a director of the learning. This may 
result in many problems and disturbances — from some teachers turning the TV set 
off, not ordering films, not fitting the material i.nto the classroom sequence of 
things, etc. The author obfjcrved an elementary class receiving TV instruction in 
the New Math and f'r?.tere 0 tecl 1n loow t-hS' taacher goiiTg to handle the con- 
tent after the presentation viJas completed. The teacher responded with consid- 
•jrable aplomb — "That’s enougli of that New Math stuff," she said, "Now let’s get 
back to OUR Mathematics — open your arithmetic Books to page 73; do exercises 



1 - 10 !" 

There are sever''! vzays in which instructional t'-ehnology can be used to 
Improi'e education. Generally, the "replacement'' system has been used, i.e, 
replace a course taught by conventional means V7ith one taught on or by TV, This 
procedure merely accomplishes the same basic purpose f./ith a little greater 
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hostil5.ty from the studexits. Some procedures ^ notably the audio-visual people^ 
use a ’’supportive" system — assist an ongoivig program to do the job a little 
better (v/hich is difficult to find validity for). Other methods do exist, but 
seem difficult for administrative, legal, or institutional means. These methods 
include concepts of repetition, time, difference, applied, review, availability. 

For example, if something i.s repeated often enough, it is more probable that 
it will be learned. Why not repeat the presentation again, over a different 
media perhaps, V7ith greater clarity for those who didn't get it 'the first time, 
or in abstract for those who v;ant merely a summary and review? It can be pre- 
seated by another person or system — programmed 5.nstruction and ETV, radio or, 

film. It can be presented during supper hours, over open circuit TV, offered at 

* 

night, dial-access, or on the telephone. One can flood the inarhet place and hit 
the students vzith many ways of presentation, many times, at their convenience. 

A large issue in this w^hole argument is one of criteria — what do we vzant the 



student to do after the instructional prograra that he couldn't do beforehand. 
Generally we keep this a secret— we don't very often meaningfully communicate to 
hlra exactly what is expected of him. Many students fail. Should V7e blame the 
student or ourselves? If there are methods which ,will improve the number V7ho 
reach the criteria, shouldn't we try it? Many times failure occurs because the 
student doesn't know what he doesn't know. Procedures can be developed V 7 hich 
could answer this problem, through the use of instructional technology. One , 
example V7ill be presented to indicate an application of instructional technology 
in areas in V7hich very little now exists in order to acknov7ledge the fact that 
instructional technology possibly v 7 ould obtain better and more usable results if 
the area of attack was a novel one rather than an already existing one. 

If V 7 e may assume that must learning V7ill be evaluated by an examination, or 
series of examlnc?.tions , and we further assume that the items on these examina- 
tions represent the content of the course, and similarly the capability of the 




stuclevit to li>dicate the degx'ce to which he has learned, then our objective 
becomes clear — i.e. to enable him to: (1) know the material \ 7 ell enough so that 
(2) he V 7 ill be able to answer correctly the ex-amination questions vze will use to 
permit him to shovr us that ho has learned the content (such questions may be 
objective, essay, performance, V7hat have you),, ,, 

The capability of the student to knov7 V7hat these test items are can cer*” 
tainly have an effect on his learning behavior. If the items are good ones and 
he knov 7 s what the items are (barring memorising of the examination for the 

s 

moment) , .his direction toward learning the correct answers to the items is the 
same as learning the content. That this is laot always true should not alter the 

I 

fact that if our assumptions are follov7ed thus far, it should be true. 

Given that he can obtain experience with these test items (takes a test, is 
graded, and learns his relative position in his class and the instructors opin- 
ion of his efforts), the feedback will inform him of what he knov7s or doesn’t 
knov7, or how much he does or doesn’t know. Such information, if presented diag- 
nostically, can offer him specific knowledge of his particular strengths and 

. « 

V7eaknesses, dependent on the feedback of v7hich items he has missed and V7hich ones 
he got right. Similarly, information could be communicated dependen .pon his 
Item results as to \ 7 here he may find a discussion about the area that that item 
•represented. 

Usually there are few rather than many examinations given during an instruc- 
tional program — the procedure takes away from instructional time, someone had to 
score them, etc. Instructional technology provides means vrhereby such problems 
need not exist. Optical Scanning equipment can read ansv;er sheets, or student 
Input stations can be used to record answers to objective questions, scored by 
computer, and fedback (by printout or video display) to indicate his evaluation. 
The student may desire many questions, or few, testing often or seldom; if the 
facilities are readily available, he may guide himself, or be directed to pursue 
a line of evaluation most necessary for him or the teacher. 



Many instructors would find such a situation intolerable because students 
may see test questions before instruction related to them has been presented, may 
be able to memorise answers to test questions, or may be able to lind right 
answers’ by ansvrering incorrectly and be corrected. This should be considered 
Improper only to the dogren that test' questions are limited in. their number, for 
If thousands of questions are available representii’ig all parts of the instruc- 
tional program, these intolerablenesses become motivational and acki'iowlcdging 
variables to relate the student to the material. Knowledge of the test, type of 
test items, and self-ai’ialysis (anonomyous) capobilitiCvS -ought to improve learning. 
The only i^roblcm is in supplying items in sufficient quavitity to form an XTBM 
BANK, vrhich the student can sample for his purposes, and from V7hich the teacher 
can sample for expiination time. 

The computer can store multiplication, division, addition, and subtrac- 
tion problems j can vary the numbers in word problems ad infinitum, can create 
questions on the basis of a test item model, can collect, file, and retrieve Ahb^ 
questions ever used on tests, Nhat is more, data on what students do with these 
questions can also be stored (how many, at what stage pass this item), and analy- 
sis infori^ation given to the student. The instructor can also obtain such data 
indicating where students are having problems. In the same manner a student can 
address the ITEM BANK with direction for V7hat kind of test item he wishes to try: 

Give me some items on photosyntheses used i.n Biology 101. 

Give me tx-/o digit division problems. 

Give me French vocabulary review items Chapter I, II, and III. 

Etc., Etc. 

Depending upon the sophistication of the instructional technology, the items 
could be visually displayed, cou3.d include graphics, charts, pictures, and confi- 
gurations, could be solved by writing, light pen, or response to a multiple- 
choice quGstD.on, The test can become a learning tool, rather than a punishment 
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03C riGgcitive )T6iufojrcing device. The computer can be programmed to evaluate vdiat 
a student asks for, how V7cll he docs, and on what he ought to be tested. Thus^ a 
student v?ho asks for a test item beyond the appropriate stage of the course can 
be automatically advised of this fact. If he gets the item incorrect, comruunica" 

tion can occur to yield appropriate statements indicating he should, perhaps, try 

' * 

item 1489. If he gets an item correct he can be praised (within the limits of 

the programmer’s skill in thinking of appropriate positive reinforceraent phrases, 

like "good," "great," "v 7 onderful,’' "very few students at your stage get this item 

right," or "Sorry — that’s not the right answer; you didn’t divide the last number 
¥ 

by 2--would you like to try another problem of this type?", etc.). 

• Another aspect of instructional technology which is interesting to observe 
is the degree to which dullness has been supported so resolutely. No inanifesta- 
tion of Madison Avenue ever crops across the ETV screen or is included in pro- 
grammed instruction or C.A.I. It is obvious from the life and death struggle on 
the commercial market and the g3,assy stares of many of the youth of America that 
commercial TV has something to offer; that commercial advertising— even for 
aspirin— does in fact work. VJhy haven’t any of these procedures and systems been 
used in educational- technology? Does commerce ask educational technologists for 
assistance in selling Volkswagens, or creating the Pepsi-generation, or creating 
an image of a political candidate? The image of education suffers by comparison. 
VJe are in the business of selling knowledge, perhaps to a much greater extent 
than we realize — learning Is not becoming an "in" thing. The establishment is 
getting criticized from right and left, and perhaps rightly so. Can V7e use as a 
viable criterion that v/e ought to be able to teach Geometry to the same extent 
that Crest can sell toothpaste? Or arc we satisfied and content that v7hat v7e arc 
doing is good and right, and thus should rest on its own laurels? 

We ’have discussed both evaluations of instructional technology and how to 
evaluate instructional technology. The picture is indeed less than adequate. 

VJhat then can be recommended? 
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One of. our major problems v;ith the use of instructional technology is the^ 
educational system itself in terms of credit allocations, managcriV-int , and aainin~ 
istrative systems, methods of showing proficiency, and product control* A system 
does not exist which gives support (i.e. promotion, salary increase) to those 
interested in improving instruction. Administrative details and incapability to 
change structures the manner and mode ’which yields us Indices of Icax'ning. The 
learning appears, sometimes, loss important than how the Individual attained it. 
In one instance we require 16 hours of a foreign language for a B.A. degree, and 
on the other hand, prevent a foreign student who is in our university studying in 
English from presenting either English or his native language as fulfillments 
for this requirement. We pass a student into the next level course if he obtains 

& ”D** or better grade in the prerequisite but we refuse to grant proficieixcy cre~ 

« 

dit to someone who already had a "D*' knowledge in that course. The system leads 
to a considerab3.e amount of ’’playing the game" rather thaix learning. Competit5,ve 
systems are not permitted. The system in fact works antagonistically to innova- 

4 

tion which includes instructional technology. 

Recommendations : 

1 , Develop funding which V7il3. lead educational institutions to build 
appropriate reward systems for the improvement of instruction- 
careers in teaching, particularly at the college level, are pre- 
requisites to improved instruction, 

2 , Supply means whereby politically separate educational institu- 

tions become interested and rewarded for producing instructional 
programs useful in more than one institution—— duplication of 
effort is wasteful and costlyj efficiency needs to become an 
educational criteria. • 

3 , Provide year long grants— in— aid for faculty to pursue reseaicn 
tov/ard the improvement of instruction j in order to improve, 
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senior personal need to be reileased for concentration on the 
teaching process. 

4. Subsiavsc large scale educational TV networks and couruumal 
C.A.I. systems; to provide means whereby institutions can 
observe presentations from other institutions, or cp use them 

cooperatively if they so desire. 

5. Develop model buildiiig educational resources, vzhose function 
it is to produce an obviously better instructional program and 
provide such proven courses nationwide—what .is lackxng in the 

# 4 

technology to date is the proof that it can offer something 
better. 

6. Support, with emphatic funditig, experimental imiovations in 
instructional programming, which will' not fail because of lack 
of personnel, writers, producers or directors, material or 

W 

equ5.pment. 

7. Study and produce an alternat5.ve administrat5.ve system oj. tne 
teacher role; to enable college departments to continue their 
graduate programs ;- 7 ithout relying on graduate teaching asslst- 
antships; to reduce' the competitive jealousy developed between 
public school teaciiers and instructional technology. 

8 . Investigate the possibility of establishing a federal free- 
university which can develop and make available courses of 
study in direct competition, or support of already existing 
courses, possibly presented over national ETV. 

9. Supply funding for the expansion of libraries to include pro- 
grammed texts. C.A.I.. and video taped or audio taped courses 
of study (remedial, advanced). 
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10. Develop systems to automate informat5.on retrieval for students, 
for single-concept film loops on long-division, to TV recordliigs 
of plays, presentations of specific points, recordings of lec- 
tures, synopses of presentations, etc. 

11. Support innovative practices v;hich deal with the management of 
education; i.e. credits, hours, requirements, grades, probation, 
required attendance, or the classroom concept. 

12. Assist in the development of procedures which vjiXl serve to 
bring the student in closer contact with that vjhich is required 
of h 3 jn — such as larger scale proficiency and diagnostic testing, 
test item bank procedures, and self-analyses systems. 

These recommendations are, of course, very general; but they are intended to 
point in the direction V7hich will en ’ le the use of technology to catch up with 
the engineering state of the art. The hardware developinenL has far outstripped 
the softv 7 are (programs, content, assimilation, structure). Most important, how- 
ever, is the system itself — it is not, and unless some radical changes are made 
In the management /administrative structure of educational institutions, V7ill not 
be able to advantageously incorporate instructional technology into the system to 
an extent which the potential demands. 
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